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Abstract - Controlling the home appliances and electronics gadgets through an Infrared remo^jo»trol is 
now in general. But the same controlling tasks can be done more easily. Primary motive of/«Mieling the 
new system of hand gesture remote control is to remove the need to look in to the hand hdftl^jflote and to 
search for a specific key for specific function. This paper presents a novel syst*» tc\control home 
appliances through hand gesture as a remote control device. The system will referaaq^^ as Handmote in 
this paper. It uses real time image processing for hand gesture recognitiorWn^jJrrared vision using 
Blobscanner [i] library and microcontroller development board , ArduinofaT^Jhis paper proposes a 
possible solution to control the gadgets for physically challenged and blind pe'jpa^ 
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l. Introductior^^^ 

Among the rising age of technology in the field of gesture recognition for hand gesture or human computer 
interaction many research are done. Here the Handmote^i^Wrred to as use of hand gesture recognition 
to control the home or office gadgets that are operateA^irough an infrared remote control in general. 
Simple remote controlled gadgets can be operatedj\8»nge a TV channel or to tune radio by finding the 
key on hand held remote control and pressing ifc^m^in this paper author puts effort to control the same 
but using hand gesture. Simply a hand gestura«or showing number of fingers TV channel can be changed or 
it can be On/Off . By showing a cross finge^o\ze?ture to the camera, TV can be muted, rotating the hand in 
clockwise or counter clockwise can chanj^yls volume level or TV channels. 




II. Proposed Concept 

There is a computer applica(i^^^signed in Java Processing^] to have real time image processing. An 
infrared camera is giving\JrajJgs in infrared vision to the computer application. After processing the 
images and recognizinCjhe hand gesture, decision data is send to a microcontroller hardware based 
on Arduino environn|3y^] . This hardware sends the data to the gadgets in same way as a remote 
control does for^e/eXruse. Gesture recognition solutions can be divided regard to the type of gesture used 
for controlling^j^^ttputer. Gesture can be considered as a change of the hand position (hand movement) 
in a particulai^dme interval with a given velocity or as a change of the hand shape (forming ellipse with 
thumb andfcMe index finger). Gesture that belong to the first group are typically called dynamic gestures 
while t/fTt^hom the second group are often referred to as static gesture. 

Software 

To implement the algorithm and its logic for run time image processing, a JAVA Processing environment [5] 
is used. Processing is a Java based programming structure. To process the images, an open source image 
processing library under GNL GPL V3 license named as Blobscanner processing library is used. Once the 
data or frame is taken from an infrared web camera. After having the image in software, code will find the 
hand based on skin detection algorithm. If nothing is available then system will be ideal but if hand part is 
detected then system will startimplementing the gesture recognition algorithm on the image to recognize 
the gesture. 
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There will be always a question of removing background noise to increase the probability of occurrence of 
correct recognition. But here Authors wants to propose a novel method for recognition environment. In the 
practice authors used an infrared camera and an infrared LED source to create bright beam of infrared light. 
This IR source is put just near to the camera in the direction of user's hand. Now whenever user will 
make a gesture, his hand comes near to camera. Hand will look more bright then other region of image 
just because camera is infrared visible. Once interested area is more brighter then background noise or 
unwanted part, it is easy to convert the image in gray-scale based on relative intensity of image. This 
setup is shown in figure i. 




Figure 1. Camera and 1R source »tuV> 
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Camera Hardware 

^l^^fendmote prototype is a simple Web-camera. The exact modification did to the camera is 
1 2. Light spectrum contains visible light and infrared light as well. Aim is to use infrared 
the IR stop filter is replaced with IR pass filter, that is why camera will not see the visible 
>ut only the IR light reflected back from hand as explain in figure i.Now this camera will give 
region images of environment to the software application. 

Algorithm 

First step after taking a frame from camera is to track the hand based on skin detection algorithm using 
Blobscanner library. Next step is converting the image into binary form and then edge tracing. Once this 
much part is ready or hand is present in image than application search for gesture behavior by counting the 
number of fingers and its orientation. Angle between two successive finger gives useful flow to recognize 
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the hand gesture. Software application reference background frame stored in variable, and every time it 
compare with upcoming frames. This gives reliable recognition. Algorithm flow is shown in figure 3. 




Microcontroller H; 



To decode the data coming from software applicatiorva^J send it to the electronic gadget, we used 
Arduino[2] development board based on microcontraH^*±mega328 from ATMEL. Processor of hardware 
is running at the speed of 16MHz. The communica*An [Between software application and Arduino is based 
Serial port. Software code uses IBM COM por^v send the data over serial port. The data rate for 
communication is 115,200 bauds/second. Ba^c^on the decision taken by software flow that which 
gesture is made, it will send a unign^^de related to a gesture. Now at hardware side, a program 
written on microcontroller will first reg^ejp^ unique code, decode it and then transmit the related data to 
gadget by means of Infrared LED safe?^ a remote control do. 
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Figure 4. Processing flow 
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III. Handmote, Proposed System 

Handmote is a prototype model to control the electronics gadgets through hand gesture. It is a wearable 
device suggest as Hand + Remote, Handmote . It consist of a computer to run software application, an 
infrared camera to take images, Arduino development board to send commands to the gadgets, IR 
torch made up of number of IR LEDs to create bright infrared beam front of hand. At this 
development stage user need to wear the device and carry a laptop bag beside back. Tentative device is 
shown in figure 7. We integrated all modules on a wearable device, so that user can wear it and interact 
with gadgets just by making a hand gesture. 



12V 
tnputer 



IR camera and Arduino board are powered up through USB port its self , rest the IR torch 
battery supply to have very bright illumination. Question is, We always need to carry a la] 
with it. Solution is briefed in section V. 

Each remote control keys has its own unique code based on manufacturer, that a^^ns'rnitted through 
modulated infrared light at 38 KHz at very low baud rate, 300-600 bauds/second^l^ram our prototype, if 
user wants to turn ON the TV, he/she will simply make predefined gesture, sofuC^S recognizes it and tells 
the microcontroller to send TURN-ON data to TV through IR LED. OtyfSfcne TV would be able to 



decode it and respond to the same, rest gadgets will neglect 
operated through remote control by Company A. rest gadget will nej 
at gadget side required, while in home automation system it i£ 
then to the gadget is shown in Figure 5. 
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Company A can only be 
Conclusively no modification 
low from software to Arduino 



Figure 5 Processing flow 
4. Testing and Experimental Results 

We tested this/PlAdmote device in both the conditions, 1. IR illumination off and 2. IR illumination on. An 
attractive r^JSjve got for 20 successive takes is here, It shows correctly recognized gesture out of 20. 
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Testing at TV showroom-B[6] 
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Testing of device for home appliances 




Testing of stereo sysl 
V. Future Work arn 




howledgement 



One disadvantage to this prototype is that, user sflbuld always keep a Laptop computer to run software 
application, however the application is writta^ivJAVA processing environment, same software application 
can be made for Mobile phone device soijilt .processing power of software will be given by Mobile phone. 
Authors are working to develop sama»fcpJcation in Android environment and making application open 
source, and available to all user J^T^at Interaction between home gadgets and human become cost 
effective. \ ^ 
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